Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fine tune PER, fix bugs #131

Merged
merged 33 commits into from
Apr 27, 2017
Merged

fine tune PER, fix bugs #131

merged 33 commits into from
Apr 27, 2017

Conversation

kengz
Copy link
Owner

@kengz kengz commented Apr 26, 2017 •
edited
Loading

Bug Fixes, Improvements

  • fix overflow error in np.exp of SoftmaxPolicy, BoltzmannPolicy by casting to float64 instead of float32
  • improve overall np.isfinite asserts
  • remove index after reset in *analysis.csv
  • remove unused specs
  • reorganize and expand test specs
  • guard continuous action value range in continuous policies
  • fix analytics param variable sourcing

DDPG

  • add EpsilonGreedyNoisePolicy

PER

  • add memory.update(errors) throughout all agents
  • add shape assert for Q values and errors throughout
  • auto max_mem_len as max_timestep * max_epis/3 if not specified
  • put the missing abs for init reward

kengz added 30 commits April 24, 2017 08:51
@kengz
use per for mountain ac
32e0634
@kengz
mountain per
aea3a3f
@kengz
fix per, add missing memory update to ddpg
763f78e
@kengz
add walker ddpg per
22c1a2d
@kengz
size down per
fc40e88
@kengz
narrow down params
a352c2e
@kengz
per for dqn v1
9cd6f61
@kengz
fix and generalize shape assert
eeba6df
@kengz
fix assert in shape
4156956
@kengz
remove offset in botlzman qstate
186cc08
@kengz
import np in sarsa
13f6f6e
@kengz
fix critic assert dim
d2d2a9c
@kengz
clipval for boltzmann at 200
e67cced
@kengz
guard overflow again
77311ef
@kengz
restore underflow fix
ef40317
@kengz
minor refactor
41f30a9
@kengz
clear out unused specs
f4cc428
@kengz
drop index col from csv
43d89ed
@kengz
mute per test
7d5e692
@kengz
fix sarsa test
668729a
@kengz
boltzman fix overflow by np float64; remove offset minus
d7f5cec
@kengz
schedule mountain dqn per
efa048e
@kengz
auto memlen for PER as 1/3 epi * timestep
8781232
@kengz
auto mem len for walker, use PER for lunar
071e13f
@kengz
fix assert size for ddpg
0457845
@kengz
reorganize tests
2ea4b34
@kengz
add more tests
21d8578
@kengz
mute atari to speed up test
96cfcbf
@kengz
guard continuous action range in policy
193ab59
@kengz
add dqn_per to start per testing
93dcb2d
@kengz kengz merged commit 4f123b8 into master Apr 27, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kengz